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Abstract 

The magnetic structure of honeycomb iridate Na 2 lr 03 is of paramount importance to its ex¬ 
otic properties. The magnetic order is established experimentally to be zigzag antiferromagnetic. 
However, the previous assignment of ordered moment to the a-axis is tentative. We examine the 
magnetic structure of Na 2 lr 03 using first-principles methods. Our calculations reveal that total 
energy is minimized when the zigzag antiferromagnetic order is magnetized along g ^ a -\- c. Such 
a magnetic configuration is explained by adding anisotropic interactions to the nearest-neighbor 
Kitaev-Heisenberg model. Spin-wave spectrum is also calculated, where the calculated spin gap of 
10.4 meV can in principle be measured by future inelastic neutron scattering experiments. Finally 
we emphasize that our proposal is consistent with all known experimental evidence, including the 
most relevant resonant x-ray magnetic scattering measurements [X. Liu et al Phys. Rev. B 83 , 
220403(R) (2011)]. 
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The 5d iridium-based transition metal oxides display very rich, interesting properties 
owing to the interplay between spin-orbit coupling, electron correlation, and crystal-field 
splitting [1-6]. In particular, A 2 lr 03 (A = Na, Li) have attracted special attentions [5-15], 
whose structure may be characterized as layered honeycomb lattices of Ir. The octahedrally 
coordinated Ir^+ ion is suggested to possess an effective jeff =1/2 pseudospin and the edge¬ 
sharing oxygen octahedron structure is proposed to realize the Kitaev model [5, 6]. As an 
exactly solvable quantum spin-1/2 system, the Kitaev model embodies Majorana fermion 
excitations and quantum spin liquid that have potential implication to quantum computing 
[16]. Although experiments have shown that the magnetic structure of Na 2 lr 03 is not a 
spin liquid but zigzag antiferromagnetic (AFM) [13, 14], the understanding of such a exotic 
magnetic structure will provide important clues to realizing Kitaev spin liquid in this family 
of materials. 

It is crucial to point out that although zigzag AFM order is well established experimen¬ 
tally, the assignment of direction of the AFM moments, on the other hand, is not without 
ambiguity. The zigzag AFM order was first proposed by combining resonant x-ray mag¬ 
netic scattering measurements and first-principles calculations, with the ordered moment 
assigned to the crystallographic a-axis [12]. In later experiments that confirmed the zigzag 
configuration with neutron scattering, the moment direction was inherited without further 
scrutiny [13, 14]. Apparently, two standing issues remain with the magnetic structure of 
Na 2 lr 03 . First, the determination of magnetic moment direction is still far from conclusive. 
There is inconsistency between the tentative experimental assignment and first principles 
calculations: previous calculations predicted that the zigzag configuration have lower total 
energy for magnetic moments along the 6-axis compared a configuration magnetized along 
the a-axis [12]. As the proposed Kitaev-like models hinges upon anisotropic interactions, the 
determination of actual direction of the AFM order parameter is clearly critical for estab¬ 
lishing reliable microscopic understanding of the low energy excitations in this compound. 
Second, and indeed, the microscopic models of Na 2 lr 03 are subject to controversy. For 
the Kitaev-Heisenberg (KH) model, which has various modifications and has been mostly 
adopted in literature [6, 7, 9, 11, 13, 20], the isotropic Heisenberg interactions do not lead 
to a special preferred moment direction while the anisotropic Kitaev interactions make the 
moments prefer the cubic f-axis of the local IrOe octahedron (to be discussed later). Several 
recent studies [29-34] analysed the necessity to adding anisotropic interactions to the KH 
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Hamitonian, which was expected to stabilize the zigzag configuration. However, the puzzle 
of magnetic moment direction assignment remains. 

In this Letter, we employ the first principles method to examine the energetics of Na 2 lr 03 , 
sampling a wide range of magnetic order with different moment alignments. Our calculations 
show that the ground state is attained in the zigzag AFM structure, with a moment direction 
g ~ a+c. We further show that the first principles energies can be well fitted with a modified 
nearest-neighbor Kitaev-Heisenberg (nnKH) Hamiltonian of spin-1/2 by adding anisotropic 
interactions, in which the Kitaev term dominates. Based on this model, we derive a few 
experimentally accessible quantities, such as the spin wave spectrum. Finally, we clarify 
that this assignment of moment direction is also consistent with resonant x-ray magnetic 
scattering measurements [ 12 ]. 

Na 2 lr 03 is a layered compound (space group C2/m)^ in which Ir ions are located at 
the center of edge-sharing octahedra formed by oxygen anions (Fig. 1(a)) [13, 14]. Thus, 
Ir ions form a honeycomb lattice within each layer. Each Ir^+ ion has five bd electrons, 
occupying t 2 g orbitals of the ideal octahedral crystal field assuming the oxygen octahedra 
remain regular. Owing to the strong spin-orbits coupling (SOC), the six Op spin-orbitals are 
further separated into two manifolds with, respectively, j^s = 3/2 and jeg = 1/2 [5]. The 
bands mainly composed of the = 3/2 states are fully filled, while the spin-orbit-coupled 
ieff =1/2 states are half filled, a keen observation that lead Khalliulin et al [ 6 ] to relate this 
material to the Kitaev’s spin- 1/2 model, with an additional Heisenberg-type interactions, in 
what is called the Kitaev-Heisenberg models. Four types of magnetic order of the j^s =1/2 
pseudospin are shown in Fig. 1(b), namely, ferromagnetic (FM), Neel AFM, stripy AFM, 
and zigzag AFM. Four theoretical models are proposed to account for the magnetic order 
in Na 2 lr 03 : (1) the nnKH model which only includes nearest-neighbor interaction between 
Ir atoms [ 6 , 7], ( 2 ) the KH-J 2 -J 3 model which also includes the second and third nearest 
neighbor Heisenberg hopping J 2 and J 3 between Ir atoms [9, 11, 13, 20], (3) the modified 
nnKH model which includes additional anisotropic interactions besides Kitaev terms and 
Heisenberg terms [29-33], and (4) the quasimolecular orbital model [18, 24, 25]. The KH 
models are based upon a local moment picture while the quasimoleular orbital model an 
itinerant picture. 

The nnKH model is the simplest model that produce the zigzag AFM ground state, with 
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(a) (b) zigzag stripy 



FIG. 1. (Color online) (a) The C2jm crystal structure of Na 2 lr 03 , viewed from slightly off the 
6 -direction, (b) Four different types of magnetic order. White and black circles denote up and 
down spins, respectively, (c) Three different types of nearest-neighbor Ir-Ir bonds. The iridium 
honeycomb plane is perpendicular to the cubic direction [111]. 

the following Hamiltonian [7]: 

H=Y. E (2A-S7SJ + JS.. Sj), (1) 

^=x,y,z <ij>ej 

where the first term is the strongly anisotropic Kitaev interaction [16] (7 = x,y, z refers to 
the three nn bonds and also the three local axes along the Ir -0 bonds of the IrOe octahedron 
shown in Fig. Ic), and the second one is the Heisenberg term. Eq. (1) can be rewritten as 
[7] H = ^ [2 sin SJ + cos^Sj • Sj), where A = \/K'^ + is a positive energy 

scale and the variety of the ” phase” angle ( tune the sign and relative strength of the Kitaev 
type and the Heisenberg type contributions in the parameter space. The anistropic energy 
for the four possible magnetic patterns, i.e., FM, Neel, stripy and zigzag, can be expressed 
as Ezigzag = f (cosC - 2sinCcos(26')), Egtripy = -E^igzag, E^u = 7 (3cosC + 2sinC), and 
-E'Neei = —Efm, where 0 is the polar angle in the local spherical coordinates of the IrOe 
octahedron. Fig. 2(a) shows that when the zigzag magnetic order is the ground state = 
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FIG. 2. (Color online) (a) Anisotropic energy of the KH model in the = 37r/4 zigzag state. The 
angle 6 is the polar angle in the local spherical coordinates of the IrOe octahedron, (b) Anisotropic 
energy in the ac-plane by first-principles calculations (solid lines) versus the total moment for the 
experimental structure of Na2lr03 . Angles are measured from the a axis. The energy of the zigzag 
order with the moment along the a axis is set to be 0. Corresponding fitted curves are also shown 
(dash lines). 

37 r/4 in the figure), the magnetic moment points along the local £-direction. This conclusion 
is consistent with the assumptions in Ref. [13]. The KH-J2--^3 model should produce the same 
qualitative conclusion on the anisotropic energy since the Heisenberg terms are isotropic. 

Motivated by the foregoing analysis, we perform detailed investigations on the anisotropic 
energy by non-collinear relativistic density functional theory, as implemented in Vienna ab- 
initio simulation package [21, 22]. The experimental structure of Na2lr03 is adopted [13]. 
The magnetic unit cell is chosen the same as the crystal unit cell, containing one layer of 
four Ir atoms, which is consistent with the consideration in the KH model. The projector- 
augmented wave potentials [23] with a plane-wave cutoff of 500 eV is employed. We use 
the Monkhorst-Pack fc-point meshes [26] of 6 x 4 x 6 per magnetic unit cell to perform the 
Brillouin zone summation. We set U = 1.7 eV, and J — 0.6 eV [28], which corresponds 
to Veff = [/ — J = 1.1 eV [17]. Such choice of result in a band gap of 341 meV for 
the ground zigzag state, consistent with the experimentally measured values (340 meV in 
Ref. [8]). We perform complete self-consistent calculations with the spin-orbit coupling 
interaction. To survey the potential energy surface of magnetization, the spin magnetic 
moment is constrained in specified directions while the magnitude is optimized. 

Figure 2(b) shows the anisotropic total energies of the four magnetic configurations in 
the ac-plane for the experimental structure of Na2lr03. The horizontal axis is the the angle 
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FIG. 3. (Color online) (a) The Ir honeycomb structure of Na 2 lr 03 and the zigzag magnetic order 
of the ground magnetic state, (b) Relative relations of the local IrOe axes x, y and z, the 
crystallographic axes a, b, and c, and also the moment direction g. 

between the total moment and the a- axis, where the total moment is the summation of the 
spin moment and the orbital moment. Surprisingly, although the zigzag state is indeed the 
ground state, the total moment points along neither the cubic f-axis suggested by the KH 
model, nor the crystallographic a-axis suggested in Ref. [12]. The energy in the ac-plane 
reaches its minimum value when the total moment points to the direction ^ ~ a + c, which 
forms an angle of 55° with the a-axis (see Fig. 3(a), where the AFM coupling between Ir 
honeycomb planes will be discussed in Supplemental Material [19]). The gf-configuration’s 
energy is signihcantly lower than the a-conhguration by about 24 meV per cell (4 Ir). 
To present the gr-direction more clearly. Fig. 3(b) shows the relative relations of the total 
moment direction g, the crystallographic axes a, b and c, and the local axes of the IrOe 
octahedron x, y and z which connect an Ir atom to one of the nearest O atoms. It is 
interesting to note that g = 2ao (1,1,0), where ao is the Ir-0 bond length, i.e., ^ is a high- 
symmetry direction of the local IrOe octahedron, [110]. The gr-direction is located in the 
cubic xy-plane and points to the middle of one 0-0 edge. The anisotropic energy reaches 
its maximum value in the oc-plane when the total moment points to the cubic £-axis. 

Figure 4 further confirms that the ^-direction is actually the moment direction of the 
ground zigzag state, consistent with resonant x-ray magnetic scattering measurement sug¬ 
gesting that magnetic moments lie in the ac-plane [12, 15]. On the other hand, the z- 
direction corresponds to the global maximum energy. To show this, the anisotropic energy 
is computed with the spin moment touring in three different planes: the ac, ab, and g^-plane 
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FIG. 4. (Color online) (a) Anisotropic energy by first-principles calculations where the angle of the 
spin moment is scanned in three different planes: ac, a6, and gb. Angles are measured from the 
a-axis to the moment direction for the ac- and a6-plane, and from the ^-direction for the ^6-plane, 
respectively, (b) Correspoinding anisotrpic energy by first-principles calculations versus the total 
moment direction (solid lines). Corresponding fitted curves by the modified nnKH model are also 
shown (dash lines). 

(see Fig. 3(b)). The scanned moment angles are measured from the a-direction for the ac- 
and a6-plane, and from the gf-direction for the g6-plane, respectively. The horizontal axes 
are the angle of the spin moment in Fig. 4(a) and the total moment in Fig. 4(b), respec¬ 
tively. The spin and orbital moments are nearly collinear, with a mutual angle less than 15°. 
As a consequence, the curves in Fig. 4(a) are similar to that in Fig. 4(b). For the zigzag 
configuration, the angle of the gf-direction relative to the a-axis is about 60° for the spin 
moment and 55° for the total moment. The total moment is ideally located in the cubic 
x^-plane of the IrOe octahedron. When the moment points along the a-axis, the energy is 
higher than that of both the b- and gf-directions. The energy with the moment pointing 
along the 6-axis is a saddle point on the potential energy surface: it is the minimum in the 
a6-plane and the maximum in the g6-plane (i.e, the cubic xy-plane). It is higher than the 
ground energy (that of the g direction) by 9.4 meV per cell. Therefore we conclude that the 
g6-plane (the cubic x^-plane) is an “easy” plane. 

Now we turn to the model explanation of the moment assignment. The prediction of 
magnetic moments along the z-axis indicates that the KH model is clearly inadequate. Here 
we show that the gf-direction assignment of magnetic moment can be explained a modified 
nnKH model with additional anisotropic interactions, where the parameters can be fitted 
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from the first-principles energies. The generalized model is described as 

E (2) 

a,(3=x,y,z <ij> 

where the 3x3 matrices Jjj on x,y,z-honds are 
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respectively. 

The form of these anisotropic exchange interactions is fixed by the assumption of perfect 
honeycomb lattice symmetry (D^ci symmetry at Ir sites), and has been reported before 
[32]. The lower symmetry of real Na 2 lr 03 crystals will in principle produce more complex 
anisotropies [29] , which we will however not consider in this work. In fitting the energies we 
treat the (pseudo-)spins Sf' as classical vectors. This model can naturally explain the zigzag 
AFM ground state without invoking further neighbor interactions. It can also produce the 
local [110] moment direction for zigzag state. The fitted curves are plotted in Fig. 4(b) 
(dash lines), with model parameters from the second column of Table I in Supplemental 
Material [19], where details of the fitting results are also presented. The fitting turns out to 
be quite good. 

From the energy dependence of moment direction for the zigzag AFM state shown in 
Fig. 4(b), we can fit the Kitaev term coefficient K, and anisotropy terms and J±±. 
The energies of other magnetic orders shown in Fig. 2(b) are required to fit the Heisenberg 
couplings. Note that although the modified nnKH model can explain the gf-direction moment 
assignment of the zigzag state, more interactions are necessary to satisfy the condition for 
zigzag ground state. The fitted curves are plotted in Fig. 2(b) (dash lines), with model 
parameters from the second column of Table III in Supplemental Material [19], where details 
of the fitting results are also presented. Our main conclusion is that the dominant interaction 
is ferromagnetic Kitaev term. 

From the fitted model parameters one can compute several experimentally relevant prop¬ 
erties. Fig. 5 shows the calculated spin-wave spectrum. It has a significant spin gap (about 
20 .8meV • S = 10.4meV) for spin-wave excitations, which can in principle be measured by 
future inelastic neutron scattering experiments. 









FIG. 5. (Color online) Spin-wave spectrum along high symmetry directions for the modified KH- 
J 2 -J 3 model under zigzag magnetic order, with parameters in the second column of Table III of 
Ref. [19]. The unit of vertical axis (energy) is meV • 5, where for ideal jeff =1/2 state 5=1/2. 
Inset depicts the Brillouin zone of the Ir honeycomb lattice. High symmetry points are r(0,0,0), 
X(7r,0,0), M(7r,7r,0), and y(0,7r,0). 

In conclusion, we have proposed an alternative moment assignment of the zigzag mag¬ 
netic order in Na 2 lr 03 using first-principles calculations. The magnetic moments are along 
the direction g ^ a + forming an angle of 55° with the a-axis, locating in the cu¬ 
bic x^-plane of the IrOe octahedron, and pointing to the middle of the 0-0 edge. The 
gf-configuration is explained by a modified nnKH model, where additional anisotropic in¬ 
teractions are included. In our picture, first-principles calculations, the modified nnKH 
model, and experimental measurements become consistent with each other. Therefore, al¬ 
though more experiments are still needed to distinguish between our gf-configuration and 
former established a-configuration, our prediction are highly probable to be supported by 
future experiments. Spin-wave spectrum is calculated, where the calculated spin gap can in 
principle be measured by future inelastic neutron scattering experiments. 

We would like to emphasize that our proposal (that magnetic moment in NalrOa lies along 
the gf-direction) is also consistent with all known experimental evidence. The most relevant 
experimental signature to the moment direction is the resonant x-ray magnetic scattering 
measurements [12], in which the original analysis on the experimental data proposed the 
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ordered moment to be along the a-axis. The same experimental data in Ref. [12] has been 
reanalyzed in Ref. [15], suggesting that the direction of magnetization makes an angle 
with the c-axis about u = 118° in the ac-plane. Since the angle enclosed by the c-axis 
and the the a-axis is /3 = 109°, which is very close to 118°, it was further proposed that 
magnetic moments were almost parallel to the a-axis. It is however crucial to realize that the 
procedure used by these authors to fit the scattering intensity does not distinguish between 
the ±C(j. The angle subtended by the c-axis and the direction of —g is 126°, which is also 
very close to 118°. Note that the moment assignment of —g is equivalent to g since the 
zigzag configuration is an AFM state. Therefore, we conclude that the gf-direction is indeed 
an alternative explanation of the experimental data. 
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SUPPLEMENTAL MATERIAL 

FM stacking versus AFM stacking of Iridate honeycomb planes 

In Fig. 3(a), the magnetic conpling between Ir honeycomb lattices is illustrated as AFM, 
according to resonant x-ray magnetic scattering measurements [SI]. However, the KH model 
neglects the weak coupling between Ir honeycomb lattices, i.e., considering only one layer of 
Ir atoms. To be consistent with the KH model, the unit cell in our first-principles calculations 
also contains only one layer of Ir atoms, which means the stacking order of Ir honeycomb 
lattices are FM. We consider such consistency reasonable since the parameters of the modi¬ 
fied nuKH model, which is adopted to explain the gf-configuration of moment assignment of 
Na 2 lr 03 in this Letter, are extracted from the results of first-principles calculations. 

For completeness, it is necessary to check the difference between ferromagnetically and 
antiferromagnetically coupled Ir honeycomb lattices. For the experimental structure, our 
first-principles calculations shows that the total energy of the antiferromagnetically coupled 
supercell is lower than that of the ferromagnetically coupled supercell by 2 meV, which 
is very small since the supercell contains 8 Ir atoms (totally 48 atoms). On one hand, 
the small energy difference confirms that the coupling between Ir honeycomb lattices is 
indeed very weak, supporting the consideration of only one layer of Ir atoms in the KH 
model. On the other hand, it explains the experimentally observed AFM coupling between 
Ir honeycomb lattices, which is the configuration for the exact ground state. Moreover, our 
calcucation shows that the total magnetic moment of the ground state for the AFM stacking 
configuration indeed points along the direction g ^ a + c. 

The relaxed structure for the AFM stacking is also checked by studying the anisotropic 


12 



FIG. SI. (Color online) Band gap versus total-moment direction. Total-moment angles are mea¬ 
sured from the a-axis to the moment direction for the ac- and a6-plane, and from the g-direction 
for the gb-plane, respectively. 


energy by first-principles calculations. The result are almost the same as that for the ex¬ 
perimental structure. Actually we find that the relaxed structure is almost the same as the 
experimental structure. While the relaxed structure for the FM stacking, the moment direc¬ 
tion of the ground state g forms an angle of 70° with the a-axis, deviating from the direction 
a -h c by 15°. Accordingly, the lattice parameters a, b, and c change slightly 1.22%, 0.99%, 
and —0.44%, respectively. The deviation of the ^-direction indicates that its sensitivity to 
structure deviations. 


Robustness of Coulomb repulsion U 

The value of Coulomb repulsion U = 1.7 eV is chosen carefully to reproduce the band 
gap provided by experiment [S2]. Fig. SI shows the anisotropic band gaps corresponding 
to different moment directions in ac-, ab-, and gb- planes for 77 = 1.7 eV. Band gap varies 
with moment direction, especially in the ac-plane. Various values of U are tested and the 
gf-configuration of the ground moment assignment turns out to be robust. 
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Fitting experimental data 


In this section we summarize our fitting results for model parameters in Eq. (2) in main 
text. Several different fitting schemes are employed. They differ in the following aspects. 

The first factor we consider is whether to treat spins Sf as constant-length-S' vectors, or 
variable-length vectors with lengths determined by the DFT obtained moment size. This 
concern comes from the fact that the moment sizes do depend on the constrained moment 
direction, and also the different magnetic orders. 

The second factor is whether to use all the data available, or only the low energy states in 
DFT calculation. The high energy states in Fig. 2(b) and Fig. 4(b), have worse convergence 
in DFT calculations compared to the low energy ones. This computational difficulty causes 
some irregularities in the energy curves in those figures. 

The last factor is whether to include second-neighbor and third-neighbor interactions. 

The fitting results are summarized in Tables I-III. Note that the fit with nearest-neighbor 
model (Table II) does not satisfy the condition for zigzag ground state. The second- and 
third-neighbor Heisenberg couplings J 2 and J 3 are thus included in the model, and their 
fitting results are presented in Table III. 



with moment size {meV/g‘^S‘^) 

normalized moments (meV/*S^) 

all data 

low energy data 

all data 

low energy data 

K 

-14.4(0.1) 

-13.2(0.2) 

-5.80(0.08) 

-4.96(0.08) 

J±± 

2 .0(0.2) 

1 .0(0.2) 

l.O(O.l) 

0.45(0.09) 

J\\± 

-1.7(0.2) 

-2.2(0.1) 

-0.78(0.10) 

-0.97(0.05) 


TABLE I. Fit to the data presented in Fig. 4(b). Here “low energy data” means data below “0 
meV” in the figure. The Heisenberg coupling J cannot be reliably fitted from these data for zigzag 
magnetic order only. Numbers in brackets are estimated error bar from the standard least square 
fit procedure. Units are m.eV/g'^S'^ if moment sizes are considered, where g is the unknown Lande 
gr-factor; or meV /if moment sizes are normalized. Ideal j^e =1/2 states will have S — 1/2 and 
g=-2. 

From the above results, we see that the ferromagnetic Kitaev interaction is always dormi- 
nant, independent with the fitting scheme we use. We believe that this is the robust con¬ 
clusion we can reach from this analysis. 
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with moment size (meV/^^S"^) 

normalized moments (meV/*?^) 

all data 

low energy data 

all data 

low energy data 

J 

7.2(1.6) 

7.2(1.1) 

2.8(0.6) 

2.7(0.5) 

K 

-11.3(2.1) 

-9.2(1.5) 

-4.9(0.8) 

-4.0(0.6) 

Jyy 

5.3(2.0) 

5.7(1.5) 

1.8(0.8) 

2.1(0.6) 

dii± 

-5.2(1.3) 

-5.4(0.9) 

-2.2(0.5) 

-2.1(0.4) 


TABLE 11. Fit to the data presented in Fig. 2(b) using only nearest-neighbor interactions. Here 
“low energy data” means data below “30 meV” in the figure. 



with moment size (meV/^^S"^) 

normalized moments (meV/*?^) 

all data 

low energy data 

all data 

low energy data 

J 

7.2(0.7) 

6.7(0.4) 

2.8(0.2) 

2.7(0.1) 

K 

-19.1(0.7) 

-16.7(0.6) 

-7.1(0.3) 

-6.4(0.2) 

Jyy 

1.5(0.8) 

1.5(0.5) 

0.8(0.2) 

0.8(0.2) 

J\\L 

-3.5(0.5) 

-3.3(0.3) 

-1.7(0.2) 

-L5(0.1) 

J 2 

-1.6(0.4) 

-0.4(0.3) 

-0.4(0.1) 

0.02(0.10) 

J 3 

7.8(0.4) 

6.4(0.3) 

2.7(0.1) 

2.3(0.1) 


TABLE III. Fit to the data presented in Fig. 2(b) with second-neighbor and third-neighbor Heisen¬ 
berg couplings J 2 and J 3 , in addition to Eq. (2) in main text. Here “low energy data” means data 
below “30 meV” in the figure. 

Some analytic results about the modified Kitaev-Heisenberg model Eq. (2) 

The classical ground states of model Eq. (2) in main text has been numerically studied 
by Ran and Kee in Ref. [S3]. Here we report some analytic results about classical ground 
state energy under the four possible magnetic ordering patterns. 


• Zigzag states: the classical ground state energy per site is 


^zigzag/ S 


2 ~ ^ Y 




'-4iL + - 2J|| 




JY 
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when the moments are along ±( 

cos( 20 z) = — 
sin(26'z) = - 


satisfies 

—AK + t/j__L — 2t/||_L 
2 -\/ 2 J||_l 

^^(-47^ +Jxx-2J||^)%8Jf^ 


• Stripy state: the classical gronnd state energy per site is 

^||± 


-AK + J±± - 2 J, 


-E'stripy/- -- - J 2 + ^ 

when the moments are along ^^^^,008^5), and 63 satisfies 

—AK + t/j__L — 2t/||_i_ 


' 11 ^ 


+ 


Jli. 


(-4ir +Jxx- 2 J||^)% 8 Jf^' 
2\/2J|l_i_ 


cos(26*s') = 
sin( 205 ) = 

^(-4ir +Jx±- 2 J||^)" + 8 J|f^ 

• Neel state: the classical ground state energy per site is 


3 7 


2 


7C + 


+ 2 J||_i_), J±± + 2 J||_i_ > 0 ; 

|(d^_L_L + 2 J||_l), J_l_l + 2J||_l < 0. 


The moments will be along 1,1) direction for the former case(Jxx + 2J||x > 0), 

ids 

for the latter case( + 2 J||± < 0 ). 


and be along ±(sindcos^, sindsin^, cosd) with (cos 2 d, sin 2 d) = 2(cos(/i+sm 0 ) ^ 

^ ^ ^ ; V , ; ^sm2 2?i+4(cos<?i+sin0)2 


E^m/S^ = Y + 3J2 + ^ + i^ + 


• Ferromagnetic state: the classical ground state energy per site is 

(J_L_L + 2J||_l), J_l_l + 2J||_l < 0; 

+ 2T||_l), + 2J||_i_ > 0. 

The moments will be along ±^^( 1 , 1 , 1 ) direction for the former case(Jxx + 2J||x < 0), 
and be along ±(sindcos^, sindsin^, cosd) with (cos 2 d, sin 2 d) = 20 ,- 2 (cos.j)+sm <p) ^ 

y sin^ 2^+4(cos ^+sin 

for the latter case(Jxx + 2 J||± > 0 ). 


From these results one can see that (1) for zigzag state energy to be lower than stripy 
state energy, we need 2 J — 6 J 3 — + 2 J||± < 0; (2) for the zigzag state to have moments 

along local (1,1,0) direction (close to the g direction in main text), we need J\\± 0, and 

—47i^ T — 2t/||_i_ ^ 0. 
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FIG. S2. (Color online) Spin-wave spectrum along high symmetry directions for the modified KH- 
J 2 -J 3 model, with parameters in the forth column of Table III. The unit of vertical axis (energy) 
is meV • 5, where for ideal jeff =1/2 state 5=1/2. Brighter region has larger spectral weight. 


Calculated spin-wave spectrum 


Spin-wave spectrum shown in Fig. 5 in main text and Fig. S2 is calculated by the 
linear spin-wave theory using the fitting parameters in the second column of Table III. The 
magnetic moment direction is determined by the solution in last section. In fact the spin gap 
Asw(^) at M point under zigzag magnetic order can be solved analytically, which reads 


Asw(M) = + B^-C^-D‘^- 2VA‘^B‘^ + 
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where 



A = -J + 3Js + ^ - J||± + ^^(-4K +J^^-2J||^)2 + 8Jf^, 
/C 3Jj__i_ — 6/^Jj__i_ + t/^_|_ + 4/^ J^||_i_ — 10Jj__Lt/||_L 

-D = --r-:-^- , -, 


4^(-4i^ +Jx^-2J||^)2 + 8J|f^ 



^ 3 -^ + 
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